The present invention relates generally to a method and apparatus for three dimensional surface contouring and ranging. In particular, the present invention uses a digital video projection system for digitally generating fringe patterns in three dimensional surface contouring.
Three dimensional surface contouring techniques have numerous applications in design and manufacturing. For example, surface contouring can be used for inspection of industrial parts whose dimensions and geometry need to be checked against their design specifications during or after manufacturing. These techniques can also be used in reverse engineering where construction of a Computer Aided Design (CAD) model from a physical part is required. In recent years, rapid prototyping technology based on a layered manufacturing concept has been established which allows for rapid fabrication of physical concept models, functional parts, and toolings directly from CAD models. Surface contouring techniques can help extend the capabilities of current rapid prototyping systems to include building physical parts and toolings from hand-crafted models or parts for which a CAD model is not available. Surface contouring techniques can also help improve the accuracy of constructed models by introducing in-process or post-process inspection into the rapid prototyping process.
Many optical three dimensional contouring methods have been developed and are well known in the art. The methods can generally be categorized into two groups: scanning and non-scanning imaging techniques. (Bieman, Leonard H., xe2x80x9cSurvey of design considerations for 3-D imaging systems,xe2x80x9d Proc. SPIE, Vol. 1005, 138-144 (1998)). The scanning techniques are represented by point triangulation (Blais, F. and Rioux, M., xe2x80x9cBIRIS: a simple 3-D sensor,xe2x80x9d Proc. SPIE, Vol. 723, 235 (1986)), laser radar (Svetkoff, D. J., Leonard, P. F., and Sampson, R. E., xe2x80x9cTechniques for real-time, 3-D feature extraction using range information,xe2x80x9d Proc. SPIE, Vol. 521, 302 (1984)),: and structured line methods. Point triangulation and structured line methods are based on the triangulation principle and the laser radar methods are based on the measurement of the travel time or phase of either a pulsed or modulated laser. All these techniques require either one-dimensional or two-dimensional scanning of the laser to cover the entire surface of the object. This generally makes the systems more sophisticated and the measurement more time consuming.
Typical non-scanning techniques include stereo vision and moire interferometry. Stereo vision obtains three-dimensional information of an object by viewing a scene from two different perspectives and then locating common features in both images. (Hobrough, G. and Hobrough, T., xe2x80x9cStereopsis for robots by iterative. stereo image matching,xe2x80x9d Proc. SPIE, Vol. 449, 62 (1983)). The processing of the images is computationally intensive, which makes the technique unsuitable for high-speed 3-D contouring.
Moirxc3xa9 interferometry is one of the most commonly used techniques for 3-D surface contouring. Compared to other techniques, it has the primary advantage of fast measurement speed due to the fact that it does not require scanning to cover the whole object surface and the image processing for extracting 3-D contour information is relatively simple. Moirxc3xa9 contouring techniques can be classified as either shadow moirxc3xa9 (Chiang, F. P., xe2x80x9cMoirxc3xa9 Methods for Contouring, Displacement, Deflection, Slope, and Curvature,xe2x80x9d Proc. SPIE, Vol. 153, 113-119 (1978)) or projection moirxc3xa9 (Khetan, R. P. and F. P. Chiang, xe2x80x9cOn the theory of two projection moire methods,xe2x80x9d Univ. Of Ill. at Chicago press, 8, 16-18 (1977); Halioua, M., Krishnamurthy, R. S., Liu, H., and Chiang, F. P., xe2x80x9cProjection moire with moving gratings for automated 3-D topography,xe2x80x9d Appl. Opt. 22, 850-855 (1983)). Shadow moirxc3xa9 uses the same grating for both illumination and observation, while projection moirxc3xa9 uses separated gratings. Another surface contouring technique is fringe projection which uses only one grating and measures surface height by triangulation.
An advantage of shadow moirxc3xa9 is that it is easy to obtain quantitative contour information from the moirxc3xa9 pattern because the grating is flat and its period known. However, the contouring of large objects is difficult because a grating with approximately the same size as the object must be used. Large gratings are difficult to make and have limited mobility.
Projection moirxc3xa9 and fringe projection offer advantages in their ability to contour large objects and the ease with which phase measuring techniques can be implemented to increase the measurement resolution. Their primary limitation is the tedium associated with obtaining quantitative height information. This limitation arises because it is necessary to calibrate both the projection geometry and the magnification factor.
In order to increase contouring resolution, phase shifting techniques developed in interferometry have been widely adopted and used in moirxc3xa9 and fringe projection methods for 3-D surface contouring. The resolution of the moirxc3xa9 and fringe projection contouring methods depends on the density of the fringe projected on the object. Generally, higher fringe density means higher resolution. However, there is a limit to the fringe density that can be applied because overly dense fringes may not be resolvable by the camera. To solve this dilemma, phase shifting techniques have been developed and widely used in optical contouring applications (Halioua, M. and Liu, H.-C., xe2x80x9cOptical Three-Dimensional Sensing by Phase Measuring Profilometry,xe2x80x9d Opt. Lasers Eng., 11(3), 185-215 (1989); Moore, D. T. and Truax, B. E., xe2x80x9cPhase-Locked Moirxc3xa9 Fringe Analysis for Automated Contouring of Diffuse Surfaces,xe2x80x9d Appl. Opt., 18(1), 91-96 (1979); Srinivasan, V. H., Liu, H. -C., and Halioua, M., xe2x80x9cAutomated Phase-Measuring Profilometry of 3-D Diffuse Objects,xe2x80x9d Appl. Opt., 23(18), 3015-3018 (1984); Srinivasan, V. H., Liu, H. -C., and Halioua, M., xe2x80x9cAutomated Phase-Measuring Profilometry of 3-D Diffuse Objects,xe2x80x9d Appl. Opt., 24(2), 185-188 (1985); Boehnlein, A. J. and Harding, K. G., xe2x80x9cAdaptation of a Parallel Architecture Computer to Phase Shifted Moirxc3xa9 Interferometry,xe2x80x9d Proc. SPIE, Vol. 728, 183-193 (1986); Kujawinska, M., xe2x80x9cUse of Phase-Stepping Automatic Fringe Analysis in Moirxc3xa9 Interferometry,xe2x80x9d Appl. Opt. 26(22), 4712-4714 (1987); Toyooka, S. and Iwaasa, Y., xe2x80x9cAutomatic Profilometry of 3-D Diffuse Objects by Spatial Phase Detection,xe2x80x9d Appl. Opt.,25(10), 1630-1633 (1986)). Phase shifting dramatically increases measurement resolution without the need of using high density fringes. Traditional phase shifting is accomplished by mechanically shifting a grating to create a series of phase shifted fringe patterns. The phase shifted fringe patterns then are processed to extract the phase of each pixel of the image using algorithms well known in the art.
Phase shifted images are generally obtained by mechanically translating a grating. The shortcomings are that the system becomes more complicated because of the introduction of moving parts into the system and the phase shifting may not be accurate due to mechanical errors. The Phase Shifting And Logical Moirxc3xa9 (PSALM) was proposed to eliminate some of the problems with traditional phase shifting techniques (Asundi, A., xe2x80x9cProjection moirxc3xa9 using PSALM,xe2x80x9d Proc. SPIE, Vol. 1554B, 257-265 (1991)). PSALM uses only one grating with the other grating generated by software in a computer. The phase-shifted moirxc3xa9 fringes are obtained through logic calculations on the image of the object and the software created grating. Since no moving parts are necessary, this technique greatly simplifies the contouring system. The problem with this technique is that the contouring result is subject to possible errors due to surface reflectivity changes and existing surface marks. Other attempts to simplify the contouring system used a Liquid Crystal Display (LCD) panel as the projection system (Asundi, A., xe2x80x9cFringe Analysis in Moirxc3xa9 Interferometry,xe2x80x9d Proc. SPIE, Vol. 1554B, 472-480 (1991); Arai, Yasuhiko, Yekozeki, Shunsuke, and Yamada, Tomoharu, xe2x80x9c3-D automatic precision measurement system by liquid crystal plate on moirxc3xa9-topography,xe2x80x9d Proc. SPIE,Vol. 1554B, 266-274 (1991)). With the creation of the fringe pattern and phase shifting being done by software, the system is flexible and simple. However, because of the low image brightness and contrast of LCD panels (Sansoni, G., Docchio F., Minoni U., and Bussolati C., xe2x80x9cDevelopment and characterization of a liquid crystal projection unit for adaptive structured illumination,xe2x80x9d Proc. SPIE, 1614, 78-86 (1991)), the quality of the fringe pattern reflected from the object is poor which causes errors in extracting surface contour information. For this reason, any meaningful contouring can only be done for small objects.
Another traditional problem associated with the phase-shifting technique is the modulo 2xcfx80 ambiguity caused by the phase extraction process using the arc-tangent function which has values only between xe2x88x92xcfx80/2 and xcfx80/2. Even though with corrections, the phase calculation range can be extended to 0 to 2xcfx80, the absolute phase still cannot be recovered. This means that if the object surface has discontinuous features, such as step-like jumps, and the height change causes a phase change exceeding 2xcfx80, then the phase extraction process cannot provide the correct height information. Accordingly, traditional phase shifting technology usually cannot be applied to measure surfaces with discontinuous geometric features. To eliminate this problem, a new technique, namely, field-shift moire, was proposed (Boehnlein, Albert, and Harding, Kevin G., xe2x80x9cField shift moirxc3xa9, a new technique for absolute range measurement,xe2x80x9d Proc. SPIE, Vol. 1163, 2-9 (1989)). Field-shift moirxc3xa9 shifts the whole projection system including the grating and the light source to capture a series of field-shifted images. With proper mathematical manipulation, both the fringe order and the phase of each pixel can be extracted to yield absolute measurement of the surface contour even for prismatic objects with discontinuous features. The problem, however, is the need to shift the whole projection system in accurate steps, which makes the system even more complicated than the traditional grating shifting technique.
When using phase shifting, at least three images, each with certain phase shift, must be taken to reconstruct the contour of the object. Traditionally, these phase shifted images are taken sequentially which limits the technique only to the contouring of static objects. In many engineering applications, 3-D surface contouring of objects under quasi-static or dynamic changes provides critically important information about the objects. In such applications, quickly capturing 3-D surface contours of objects becomes necessary. There have been some reports on using color as a means to boost contouring speed while keeping the resolution intact. K. G. Harding xe2x80x9cColor encoded moire contouring,xe2x80x9d SPIE Proc. Vol. 1005 Optics, Illumination, and Image Sensing for Machine Vision III, 169-178 (1988) and European Patent No. EP0076866, Harding proposed a color-encoded moirxc3xa9 technique that retrieves the 3-D surface contour of an object from a single snap shot of the object illuminated by a color-encoded fringe pattern. Contouring speed was limited only by the frame rate of the camera. However, since the color-encoded fringe pattern produced on a Polaroid film had a poor contrast ratio, no actual contouring of objects was attempted.
European Patent No. EP0076866 discloses the simultaneous projection of three color-coded patterns on an object with the patterns being phase-shifted by 120 degrees relative to each other. The grating patterns deformed by the object surface are recorded corresponding to their color coding by three associated color-selective cameras. As a result of this, the pattern is recorded simultaneously in three different phase relations so that an evaluation based on phase shifting algorithms may be performed without requiring a mechanical shifting.
International Publication Number WO 99/34301, which was developed by two of the coinventors of the present invention, discloses a three dimensional surface contouring method and apparatus based on the full-field fringe projection technique. A digital video projection system is used to project digitally created fringe patterns onto the object. The fringe pattern distorted by the geometry of the object surface is then captured by a high resolution CCD camera. To increase contouring resolution, purely software-based digital phase shifting technique is used, which eliminates the need for accurate positioning systems in the traditional phase shifting methods. The surface is reconstructed by applying the phase wrapping and unwrapping algorithms. The disclosure of International Publication Number WO 99/34301 is incorporated herein by reference.
Each of the methods and apparatus described above has inherent shortcomings which detract from their use in three dimensional imaging, and it is an object of the present invention to overcome such shortcomings and to provide an improved method and apparatus for three dimensional surface contouring.
The present invention is a method and apparatus for three dimensional surface contouring and ranging. The present invention uses a digital video projector for projecting fringe patterns in surface contouring.
The method of three dimensional surface contouring of an object having a. surface defining a geometry includes generating a plurality of phase shifted digitally-interpretable fringe pattern signals with each signal being generated at a separate phase angle. The signals are then converted into optical phase shifted fringe patterns which are projected onto the surface of the object. The geometry of the object distorts the fringe patterns. A reflection of each of the distorted fringe patterns is individually retrieved. The distorted fringe patterns are combined to generate a phase-wrapped image. The phase-wrapped image is unwrapped to generate a relative phase map of the surface of the object. An absolute-phase mark is projected onto the surface and the reflection of the absolute-phase mark is retrieved. An absolute phase map is generated by translating the relative phase map. Preferably the coordinates of the surface of the object are obtained by transforming the absolute phase map.
The apparatus for three dimensional surface contouring of an object includes a signal generator that generates an absolute-phase mark signal and a plurality of phase shifted digitally-interpretable fringe pattern signals with each of the signals being generated at a separate phase angle. A digital video projector receives the signals from the signal generator. The digital video projector converts the fringe pattern signals into optical fringe patterns and the absolute-phase mark signal into an absolute-phase mark. The digital video projector projects the fringe patterns and absolute-phase mark onto the surface of the object. The fringe patterns are distorted by the geometry of the object and an optical retrieval device retrieves a reflection of the distorted fringe pattern and a reflection of the absolute-phase mark. An image generator combines the distorted fringe patterns and reconstructs a relative phase map of the surface of the object and translates the relative phase map to generate an absolute phase map. Preferably the image generator is configured to transform the absolute phase map into coordinates of the surface of the object.
In a preferred embodiment of the invention, at least three phase shifted fringe patterns are generated separated by 120 degrees. The phase shifted fringe patterns can be projected sequentially. Preferably the sequential projection of the phase shifted fringe patterns is synchronized to increase contouring speed. Contouring speed can also be increased by projecting a plurality of phase shifted fringe patterns substantially simultaneously by color encoding the phase shifted fringe patterns. The absolute-phase mark can correspond to only one pixel. Preferably the absolute-phase mark is a line. The absolute-phase mark can be projected simultaneously with at least one of the fringe patterns. Preferably the absolute-phase mark is color encoded. Preferably where there are a number of phase shifted fringe patterns, the phase shifted fringe patterns are separated by the quotient of 360 degrees divided by the number of phase shifted fringe patterns.
As a result of the present invention, a method and apparatus for three dimensional surface contouring and ranging is provided. A particular advantage is that since the fringe patterns are generated digitally and projected by a digital video projector the fringe patterns have exceptionally high brightness and contrast ratio. In addition, since the fringe patterns are generated digitally, fringes with any cross-sectional intensity profile and spacing can be produced. Further, the digitally controlled phase shifting technique eliminates the traditional need for physically shifting a grating or other optical components which translates into higher contouring accuracy. Moreover, fringe patterns can now be easily color encoded for a variety of applications.
For a better understanding of the present invention, reference is made to the following description to be taken in conjunction with the accompanying drawings and its scope will be pointed out in the appended claims.